The Perversity of Measuring Trust
I once consulted to a convenience store chain with 150% store manager turnover. Turns out they gave lie detector tests to each manager every month to cut down on theft. The frequent measurement sent a message to managers— “someone must be getting away with this, maybe I’ll try it.” Click. Results perverted.
The Gallup Management Journal has a piece titled “Engaging Customers—All Day, Every Day.” Here’s how they tee up the issue:
It’s tough to exaggerate the importance of customer engagement. Fully engaged customers deliver a 23% premium over average customers in share of wallet, profitability, revenue, and relationship growth, according to Gallup research, while actively disengaged customers represent a 13% discount on the same measures…
So, considering the amount of money on the line, it pays to scrutinize every single customer-employee encounter. That’s exactly why so many call centers survey customers to determine their level of engagement.
But what happens between measurements? At best, customer service representatives (CSRs) see their customer scores once a month. Many CSRs only get their scores quarterly or annually. It’s difficult to sustain energy and focus on customer engagement when the feedback is infrequent. The challenge for team leaders and managers becomes finding ways to keep CSRs committed to engaging their customers, even when new data isn’t available. Specifically, how can team leaders keep customer engagement from feeling like an isolated event, rather than a way of doing business?
Note two assumptions the article makes:
1. The purpose of all this “customer engagement” stuff is to increase share of wallet, profitability, revenue, and relationship growth for the seller;
2. The ideal management tool for accomplishing the above would be massive feedback in the form of customer survey data—way more frequently than monthly.
This is perverse on so many levels I don’t know where to start. Let’s look at just three.
Perversity number one: the advice that Gallup then serves up is excellent—and has almost nothing to do with their premise. Their advice is to establish emotional connections with customers across four levels of customer engagement—Confidence, Integrity, Pride, and Passion—and they give excellent advice about how to do it. Well done.
Why oh why, then, do they position this advice as being the fallback alternative to multiple choice phone surveys? When was the last time you filled out a “how did we do” survey except when you were pissed off? The data are necessarily non-specific, post-hoc, and suspect. Why do we think it better than real-time good supervisory feedback?
Perversity number two: mistaking the end for the means. If integrity is in service to making a buck, why should anyone trust you? Faking sincerity is a con job.
No one knowingly trusts a con man—because he’s in it for himself. Companies who say the purpose of customer service is to make money are simply con men who are more up-front about their goal.
Make profit an outcome, not a goal. If you make it a goal, you sow distrust. If your goal in establishing trust is to make money, you will lose both.
Perversity number three: believing that extrinsic measures are better than intrinsic ones. If the system rewards and encourages me for going from a weekly score of 3.5 to one of 3.6, then I will obsess about tricks, tweaks and tunings. If you then peg my compensation to it, you can be sure the last thing I’ll think about is true customer connection. Click. Results perverted.
Read Alfie Kohn’s fine book Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise and Other Bribes. Someone who actually cares about me will beat the pants off someone who fakes caring to make money off me. And ironically, the first person is far more profitable anyway. As a byproduct.
Do what Gallup recommends as second best; it beats the hell out of massively focusing on extrinsic measures. And not just for convenience store managers.
Charlie,
I agree with your thought that if your motive for increasing trust is to make money you’ll not do as well as the more sincere person who really cares about the relationship, all other parameters being equal.
However, isn’t it more precise to recognize that the platform upon which one attempts to build this trust is the profit which is the initial cause and ultimate result of the relationship?
I believe that the level of trust in the relationships I forge with my clients has a very material effect on the length as well as the sales growth in those relationships but we all know that the reason I called in the first place was to secure profitable business and that without that the relationship would die.
A Comment about Perversity #3:
Many companies tie pay and/or bonuses to "performance metrics". I’ve worked with Customer Service groups who reward their employees based on their average length of the call with customers…the shorter the average, the better bonus for the employee.
You can bet that there aren’t that many employees interested in actually listening and/or helping to the customer…they are more interested in trying to get off the phone as quickly as possible. Some even hang up on callers to keep their average low.
Great post Charles.
The retail outfits where I enjoy shopping are those that put that Confidence, Integrity, Pride, and Passion into action. For me, those are Borders, Whole Foods, LL Bean, and Roche Bros. (local Boston grocery chain). Never do the employees act as if it’s a drag or inconvenience to deal with me.
When I think back on the tech companies I’ve worked in, I realize that it doesn’t work if you don’t have all four AND the confidence and pride in your products and services are well-founded. I worked in a software company that lived on well after it should have died because we had Confidence, Integrity, Pride, and Passion – but it was mostly around our ability to survive in the face of the market reality that our technology (which originally well-wrought and still useful) was outmoded, and we had no financial ability to give it the complete overhaul it needed. We worked overtime to make product tweaks and to do whatever we needed to do to support our customers. But in the end, the ugly reality caught up with us and we got acquired and subsumed.
As usual, your post got me thinking!
I certainly nod my head at the companies Maureen selects. And I also recognize that what Eric says is depressingly common. It is a source of continued amazement to me that the admiration of LL Bean et al is so widespread, yet the practices Eric talks about are so prevalent.
Phil McGee, it seems to me, is absolutely correct in adding precision. Business is not charity, a corporate entity is not an
eleemosynary institution. I certainly don’t recommend pulling the plug on a conversation the moment you think the probability of sale crossed a line, but it’s also true that we don’t do what we do just to be nice.
Maybe the way to put it is that we’re all in business for several reasons, profit being a big one, but that things—including profit—go very much better when we let it be a byproduct as well as a goal, when we manage it in longer timeframes than we measure it, and when we operate from integrity.
I’m a bit of a cynic, but I do believe that bit about doing well by doing good. Business done with the customer in mind is better business. And no harm in saying it’s all business.
Phil?
I think too many companies do not follow the common sense advice of, "Make profit an outcome, not a goal." Many companies are so focused on short-term profit that they lose sight of what it actually takes to earn one. It’s rare to see a company with a long-term customer strategy, but those that have it end up with engaged customers and profits to boot. It doesn’t have to be one or the other.
money/profit vs. me/customer…
why in our culture does it seem to be an "either/or"propositon rather than a "both/and" proposition…
what’s the deeper thinking (underneath "dollars" and/or underneath "people/relationship") that brings one to support one proposition or the other…
Eric,
I’ve been there. The incentives become very… perverse. I call it "putting your fingers down". Management can put its fingers down wherever it wants and get the behaviour it measures and rewards, but they better be sure they know what they really want because the natural flow of the job will rearrange itself, usually in ways they won’t like.
Mr. Green,
I’m a regular reader of the Gallup Management Jounral. I had a much differerent reaction to the article.
"Perversity One" – I didn’t perceive them to say establishing emotional connections with customers was a fallback to "phone surveys." My take was the author was merely providing very good advice to call center managers on how to help thier CSR’s maintain focus on engaging with customers. His point, I think, is that we all tend to focus more on certain things when we know we’re being measured on them. People tend to change their behavior patterns temporarily when they’re being measured or feel eyes are on them.
A good example is sales training that appears to be effective for a short duration simply because sales reps temporarily adjust their behavior patterns when they know management is paying more attention. With a number of weeks, however, nearly all sales reps will fall back to previous patterns. In this article the author is offering advice to managemers of how to maintain desired behavior patterns and focus for their CSRs in between thos measurement periods.
Perversity Two – It’s a nice thought that we should make profit an outcome, not a goal. However, I don’t know too many business professionals who have not included some form of profit (e.g. revenues, # of clients) as their primary goal for 2007.
And is there really such a thing as a "con man?" And is there really such a thing as an "honest man?" I don’t question that we all have labeled or perceive certain individuals to be one or the other. According to a number of studies, though, we all use lying and deception on a regular basis. If you believe those studies, then, we’re all capable of being "con men." Obviuosly we coulcd argue about frequences and "degree" of lying. Furter, studies show lying is used by individuals based on situational factors. Situation factors, then, determine if deception is used, not if someone is inherently a "con man" or not.
Perversity Three – Again, didn’t think the author meant this at all. If fact, if you read all of Gallup’s material, their top advice would probably suggest hiring CSR’s who intrinsically have the talents for establishing customer engagement. A company like Maritz, would be much more apt to tout what you say as far as believing extrinsic measures are better than intrinsic ones.
I’m not associated with Gallup at all, but referring to the article as "perverse on so many levels" seems extreme to me. Of course, great way to garner comments. Just my two cents.
Rob,
Thank you for your very thoughtful and well-articulated comments; I do appreciate your taking the time to contribute here.
Let me say I may come to regret the phrase "perverse in so many ways." You’re the second to comment on it. If I get one more, I’ll go back and edit it out, and say why I’ve done so in a comment. I do not want to be the source of what is perceived as an ad hominem attack on Gallup.
I very much share your admiration for Gallup; they are generally excellent in so many ways that it struck me as perverse that they would deviate, as I saw it, in this case. In fact, I said in the post, "the advice that Gallup then serves up is excellent." Later in the post I said "they give excellent advice about how to do it. Well done." So let me underscore again—Gallup does great work. I find this piece an anomaly, not a pattern.
Let me respond to your comments.
You, I and Gallup seem to be in agreement that the article is about what should be done between measurements. Gallup puts it this way: "what happens between [customer survey] measurements?" You put it: "how to maintain desired behavior patterns and focus for their CSRs in between those measurement periods." I suggested they were positioning engagement as something you focus on to bridge the times between measurements.
Still sounds like "fallback" to me. The implication is, asI read it, if you could measure all interactions all the time, that would be the ideal. I’m trying to suggest that that belief can—and often does—lead to bad customer interactions.
Regarding profit as an outcome, I too have annual goals for revenue, profit and clients. In no way am I arguing against that. In fact, I’m not even arguing against short-term measurements. I’m arguing against the endemic belief that the best way to affect short-term metrics is short-term management. I’m trying to argue that the best short-term performance comes from long-term management, not short-term. The measures themselves are not harmful per se, they’re not the issue.
I agree with you completely about the fascinating continuum between full truth-telling and total lying. We all inhabit various parts of that spectrum at different times. I agree with you about things being situational, and in fact I never said anything about anyone being "inherently" a con man; as you said, we are all indeed capable of it. No argument.
(I suggest anyone interested in this read Sissela Bok’s excellent book, Lying: Moral Choice in Public and Private Life. To unjustly distill it down, you cannot make the case for a complete prohibition against lying; but you can almost always tell more truth than we are inclined to do. The world does not suffer from too little lying, but from too much.
Regarding my "perversity number three," you say the author didn’t at all mean what I said, namely arguing for extrinsic motivators over intrinsic ones. You are right, he did not literally say that rewards ought to be specifically linked to measurements. Though I’d be surprised if he didn’t meant that.
I’m sure that, as you suggest, Gallup does recommend hiring people who are innately inclined toward customer engagement. But as I pointed out in my wraparound example: the convenience store chain I consulted to also tried very hard to hire honest people. They actually defined their problem as one of finding better hiring criteria to find more honest people.
But that was not their problem. The problem was that their frequent measurement systems actually drove honest people to theft. If that doesn’t deserve the name perversity, I don’t know what does.
I think the real difference between us is this. Both you (at least in this comment) and Gallup (in this one article as I read it) seem to believe that measurement improves performance. This is the Hawthorne effect, first observed in the 19th century. It is still evident today, in sweeps weeks on TV, and in call centers. You can easily find a lot of examples where people behave better because they’re being watched. As you put it, "People tend to change their behavior patterns temporarily when they’re being measured or feel eyes are on them."
Clearly you believe their performance improves, not declines, by being measured. And the clear import of the Gallup article is the same thing. Further, the article seems to suggest the more the better. If some data improves performance, then more data will improve it even more. Because we don’t have enough customer data often enough, we have to rely on fallback methods like the ones they describe—the sort of coaching that I actually think is far superior to frequent discussions about short-term data.
I would argue that that leads to short-term-ism. You can take the most internally motivated people in the world and if you give them monthly or weeklyor daily metrics, if you subject them constantly to discussions about short term metrics—and tie in rewards to boot—you will produce rats pecking the bar to get food, who have lost touch with the underlying reason you put the metrics in place in the first place. The measures become the goal. You get short term behavior.
Short term behavior is very desirable in certain businesses—for example, retail stores have extremely detailed weekly and daily data, because when it rains, you want umbrellas on hand to sell. For example, the airlines need detailed statistics to peform load management and pricing. In those businesses, short-term metrics are there for a good business reason—you need short term management.
But for most businesses, when you put your customer relationships in the hands of people who are constantly subjected to that kind of short term information, the temptation is almost always to default to short term behavior. Link that short-term behavior to money, and you get perversity.
(An example from Alfie Kohn’s book: a study observed children playing games. The researcher then offered the kids rewards for playing their favorite game. The result was they lost interest in playing the game. Click. Perversion.)
The best short-term results do not come from short-term management. They come from long-term management. Data can help, but it is not the answer. Focus on short term data without long-term context breeds perversion. Using lots of short term data to have long term discussions is great.
I’m not an expert on CSRs and call centers, though I do know something about managing professionals, so here’s a generalization. Consulting irms that focus on weekly utilization have worse customer satisfaction ratings than those that focus on it monthly. And consulting firms with healthy customer satisfaction ratings are more profitable. The focus on short-term metrics actually causes short-term performance to fail.
People who focus on the quality of the customer relationship for the sake of the relationship have higher customer sat ratings that those who focus on the customer sat ratings for the sake of the ratings. After a certain level of saturation, the Heisenberg Principle overwhelms the Hawthorne effect. People who are measured all the time will go passive aggressive on you, resisting in subtle ways and taking it out on the customer. Check the comments above this one from Ian Welsh, Liz Glaskowski and Eric Brown who I think speak to that effect.
I’m rambling. Thanks again Rob for a thoughtful post and please feel free to come back at this one.
Check out the trust index of brands, service providers, people, institutions, etc at http://www.trust-index.com
<a href="http://www.trust-index.com">Trust-index</a>
You can add any item you want to be rated by people from all over the world. Express your trust, and check out what others think about.
The comment just above here, from "Antuan" dated Dec. 23,2008, is pretty clearly spam.
I’m going to leave it stand for a bit, though, as a powerful example of perversity in measurement.
Here is a site purporting to give trust "ratings." The idea is, anyone can log in and rate anything or anybody. Anyone interested in finding out someone’s trust "ratings" can look them up.
This is colossally untrustworthy. Another attempt at the same thing was Rapleaf. Both practically suborn false self-ratings by those who see some interest in being rated as "trustworthy." It’s an open invitation to fudge data and behave in an untrustworthy manner.
"Antuan," if "you" want to write back and defend this shameless attempt at self-promotion, feel free. Meanwhile, let me just state publicly it’s crap, and spamming it onto blog posts is exactly the appropriate publicity strategy for such garbage.
"Charlie", as you request, I write to defend my position.
As you put it:
"The idea is, anyone can log in and rate anything or anybody. Anyone interested in finding out someone’s trust "ratings" can look them up."
It is true, that is exactly the aim of Trust-index. You don’t even need to login to rate items.
You also mention that "This is colossally untrustworthy". Well, this index is not meant to give 100% accurate measures. The goal is to have another point of view, not being the one and only. This is explained in the FAQ.
So, you might be thinking, what measures have been taken to provide the highest possible accuracy?
* only one vote per IP number: this implies that no two anonymous persons can vote for the same item from the same computer. That is, you cannot vote indiscriminately for the item you have interest in. Unless, of course you go computer by computer voting, which takes time, trouble and will not modify the trends given if the item has been voted a large number of times.
* a registered user cannot vote more than once for a particular item: this implies that if you want to vote several times, you will need to register yourself with different emails. Again, this takes time, trouble and will not modify the trends over a large number of votes.
As I have put it at the beginning mathematical accuracy is not the goal. We just want to provide a measurement that can complement others.
About suborns… well, what can I say about this? As you will notice, there is no money mentioned anywhere in the website. You can not pay to achieve a better index or to add or remove votes.
Best regards
antuan
Antuan, thanks for writing; first of all, I’m glad to see you exist, and are quick and responsive. That’s all good.
I hear you about one vote per person and no money; however, I have a lot of other concerns.
Take a look at the saga of Rapleaf, well-stated at here.
Rapleaf was much like what you’re trying to do, I think. (I notice that the latest version of Rapleaf has completely bypassed the trust ratings issue, perhaps because of the failure of the ratings approach for precisely the same reasons I"m citing here).
The core two flaws are allowing anonymous people to rate (e.g. you can rate yourself), and self-orientation (blatant selling of lists by RapLeaf).
Can you explain why this trust rating is different from RapLeaf? In particular:
1. What’s to prevent me spamming multiple times from different addresses rating me or my company highly?
2. What’s your business model, and does it depend in any way on selling the ratings? If so, what’s to prevent the buyers of the data from also being contributors to the ratings themselves? That’s another aspect of what I mean by suborning?
Charlie, I don’ t know very well how Rapleaf works so I will try to answer your questions without making any comparison:
>> 1. What’s to prevent me spamming multiple times from different addresses rating me or my company highly?
As I stated in my previous post there isn’t as of now a mechanism preventing that you do rate the same item several times. However I have tried to make it "uncomfortable" so to say. You will need either to change from one computer to another on each vote, or register different users with different emails, verify them all, then login and vote with all of them one by one. As you can see it is not foolproof but in order to forge the results you will need quite some time and effort. And, if the number of votes is big enough statistically, you might not be able to change the trends anyway.
To try to make it even better, we are thinking about applying a new algorithm in the near future for the calculation of the Trust-index. Right now it is a simple arithmetic mean value of all votes. The idea of the new algorithm would be to assign each vote some "weight". This weight should be calculated based on:
* a registered users’ vote will have more weight if the average of all the votes he has issued for the different items is close to the central point. This will make less important the votes issued by very optimistic or pesimistic people. Also it will sort of discard a little the kind of fraudulent tactics that you mention.
* anonymous votes will also be weighted less. Somehow we believe that a vote issued by a registered user that has a verified email address should be more favourably valued than anonymous ones.
However this is just the idea, it is not yet implemented. Suggestions are accepted! 😉
>> 2. What’s your business model, and does it depend in any way on selling the ratings? If so, what’s to prevent the buyers of the data from also being contributors to the ratings themselves? That’s another aspect of what I mean by suborning?
About the business model. There is no business model since Trust-index is not supposed to be a business. You can consider it a kind of webdesign experiment. For us it is a place to test some web techniques and also to extract some market research data. At some point in the medium term we might be considering adding some google advertisement but in no case will we sell ratings, votes, or anything that could allow forging the indexes.